Recent Trends in Comparability Studies

The purpose of this paper is to review the research addressing the comparability of computer-delivered tests and pencil-and-paper tests, and particularly the research since 1993. The first part of this paper summarizes the state of online testing technology and the different methods used in the comparability studies. The second part discusses the results from the studies, specifically in K-12 testing. The last part discusses the potential of online assessments. Research in K-12 education shows that students are using computers in school for classroom-based activities as part of their everyday learning (U.S. Department of Commerce, 2002). In addition, the disparity in computer access among K-12 students has been shown to be negligible over the past five years (Kleiner & Lewis, 2003; Parsad, Jones, & Greene, 2005). Studies indicate that K-12 students are familiar with computers and feel comfortable using them (DeBell & Chapman, 2003; Higgens, Russell & Hoffmann, 2005; Poggio, Glasnapp, Yang, & Dunham 2005; O’Malley et al., 2005; Ito & Sykes, 2004). With respect to specific comparability research, evidence has accumulated to the point that it appears that the computer may be used to administer tests in many traditional multiplechoice test settings without any significant effect on student performance. One exception to this finding is with respect to tests that include extensive reading passages; for these, studies have tended to show lower performance on computer-based tests than on paper tests (Mazzeo & Harvey, 1988; Murphy, Long, Holleran, & Esterly, 2000; Choi & Tinkler, 2002; O’Malley, Kirkpatrick, Sherwood, Burdick, Hsieh, & Sanford, 2005). These differences may be due to issues related to scrolling and the strategies that students use to organize information (e.g., underlining key phrases). As students continue to become more familiar with reading on the computer and as computer interfaces begin to include tools to enhance student’s reading comprehension, these differences may disappear.